Introduction to Computer Vision: Plant Seedlings Classification¶
Problem Statement¶
Context¶
In recent times, the field of agriculture has been in urgent need of modernizing, since the amount of manual work people need to put in to check if plants are growing correctly is still highly extensive. Despite several advances in agricultural technology, people working in the agricultural industry still need to have the ability to sort and recognize different plants and weeds, which takes a lot of time and effort in the long term. The potential is ripe for this trillion-dollar industry to be greatly impacted by technological innovations that cut down on the requirement for manual labor, and this is where Artificial Intelligence can actually benefit the workers in this field, as the time and energy required to identify plant seedlings will be greatly shortened by the use of AI and Deep Learning. The ability to do so far more efficiently and even more effectively than experienced manual labor, could lead to better crop yields, the freeing up of human inolvement for higher-order agricultural decision making, and in the long term will result in more sustainable environmental practices in agriculture as well.
Objective¶
The aim of this project is to Build a Convolutional Neural Netowrk to classify plant seedlings into their respective categories.
Data Dictionary¶
The Aarhus University Signal Processing group, in collaboration with the University of Southern Denmark, has recently released a dataset containing images of unique plants belonging to 12 different species.
- The dataset can be download from Olympus.
- The data file names are:
- images.npy
- Labels.csv
Due to the large volume of data, the images were converted to the images.npy file and the labels are also put into Labels.csv, so that you can work on the data/project seamlessly without having to worry about the high data volume.
The goal of the project is to create a classifier capable of determining a plant's species from an image.
List of Species
- Black-grass
- Charlock
- Cleavers
- Common Chickweed
- Common Wheat
- Fat Hen
- Loose Silky-bent
- Maize
- Scentless Mayweed
- Shepherds Purse
- Small-flowered Cranesbill
- Sugar beet
Note: Please use GPU runtime on Google Colab to execute the code faster.¶
Importing necessary libraries¶
# Installing the libraries with the specified version.
# uncomment and run the following line if Google Colab is being used
!pip install tensorflow==2.15.0 scikit-learn==1.2.2 seaborn==0.13.1 matplotlib==3.7.1 numpy==1.25.2 pandas==1.5.3 opencv-python==4.8.0.76 -q --user
Note: After running the above cell, kindly restart the notebook kernel and run all cells sequentially from the start again.
import os
import numpy as np # Importing numpy for Matrix Operations
import pandas as pd # Importing pandas to read CSV files
import matplotlib.pyplot as plt # Importting matplotlib for Plotting and visualizing images
import math # Importing math module to perform mathematical operations
import cv2 # Importing openCV for image processing
import seaborn as sns # Importing seaborn to plot graphs
from sklearn import metrics
# Tensorflow modules
import tensorflow as tf
from tensorflow.keras.preprocessing.image import ImageDataGenerator # Importing the ImageDataGenerator for data augmentation
from tensorflow.keras.models import Sequential # Importing the sequential module to define a sequential model
from tensorflow.keras.layers import Dense,Dropout,Flatten,Conv2D,MaxPooling2D,BatchNormalization # Defining all the layers to build our CNN Model
from tensorflow.keras.optimizers import Adam,SGD # Importing the optimizers which can be used in our model
from sklearn import preprocessing # Importing the preprocessing module to preprocess the data
from sklearn.model_selection import train_test_split # Importing train_test_split function to split the data into train and test
from sklearn.metrics import confusion_matrix # Importing confusion_matrix to plot the confusion matrix
from sklearn.preprocessing import LabelBinarizer
from keras.applications.vgg16 import VGG16
# Display images using OpenCV
from google.colab.patches import cv2_imshow # Importing cv2_imshow from google.patches to display images
from sklearn.model_selection import train_test_split
from tensorflow.keras import backend
from keras.callbacks import ReduceLROnPlateau
import random
# Ignore warnings
import warnings
warnings.filterwarnings('ignore')
Loading the dataset¶
from google.colab import drive
drive.mount('/content/drive/')
#loading data into a pandas dataframe
path="/content/drive/MyDrive/Great_Learning/files/Computer Vision/project_5/Labels.csv"
Labels_df=pd.read_csv(path)
# Load the image dataset (assuming it is a NumPy array)
image_path = '/content/drive/MyDrive/Great_Learning/files/Computer Vision/project_5/images.npy' # adjust the path as needed
images = np.load(image_path)
Data Overview¶
Understand the shape of the dataset¶
print(images.shape)
print(Labels_df.shape)
# Displaying the shapes of the loaded datasets to verify the data
print("Shape of Labels DataFrame:", Labels_df.shape)
print("Shape of Images Array:", images.shape)
Observations:¶
There are 4750 RGB images of shape 128 x 128 X 3, each image having 3 channels.
Plotting images using OpenCV and matplotlib¶
cv2_imshow(images[5])
plt.imshow(images[5])
Observations:¶
OpenCV cv2.imshow:
Displays the image in its default BGR format (colors may look incorrect as OpenCV does not convert BGR to RGB).
Image appears smaller and uses a pop-up window.
Matplotlib plt.imshow:
Displays the image correctly in RGB format.
The image appears larger and integrates well within Jupyter Notebook.
It shows axis scales, which are customizable.
Convert the BGR images to RGB images.¶
# Converting the images from BGR to RGB using cvtColor function of OpenCV
for i in range(len(images)):
images[i] = cv2.cvtColor(images[i], cv2.COLOR_BGR2RGB)
BGR to RGB Conversion Observations:¶
Why Conversion is Needed:
OpenCV loads images in BGR format by default.
Most deep learning frameworks (like TensorFlow, PyTorch) and visualization libraries (like Matplotlib) use the RGB format.
Code Explanation:
cv2.cvtColor(images[i], cv2.COLOR_BGR2RGB) converts each image from BGR to RGB.
A loop is applied to process all images in the dataset.
Impact:
Ensures proper color representation for visualization and training.
Prevents color channel mismatch issues during training.
Efficiency:
- Looping through the dataset works, but for larger datasets, consider vectorized operations or optimizations for speed.
Exploratory Data Analysis¶
- EDA is an important part of any project involving data.
- It is important to investigate and understand the data better before building a model with it.
- A few questions have been mentioned below which will help you understand the data better.
- A thorough analysis of the data, in addition to the questions mentioned below, should be done.
- How are these different category plant images different from each other?
- Is the dataset provided an imbalance? (Check with using bar plots)
def plot_images(images, labels_df):
"""
Plot random images with their corresponding labels.
Parameters:
- images: NumPy array of images (assumed shape: (num_images, height, width, channels)).
- labels_df: Pandas DataFrame containing the labels with 'Label' as the column name.
"""
rows = 3 # Number of rows
cols = 4 # Number of columns
fig = plt.figure(figsize=(10, 8)) # Figure size
for i in range(rows * cols):
random_index = np.random.randint(0, len(labels_df)) # Random index
ax = fig.add_subplot(rows, cols, i + 1) # Add subplot
ax.imshow(images[random_index]) # Plot the image
# Set the title as the corresponding label
ax.set_title(labels_df['Label'].iloc[random_index])
plt.tight_layout()
plt.show()
plot_images(images,Labels_df)
1. How are these different category plant images different from each other?¶
def plot_category_representatives(images, labels_df):
"""
Plot one representative image for each category.
Parameters:
- images: NumPy array of images (assumed shape: (num_images, height, width, channels)).
- labels_df: Pandas DataFrame containing the labels with 'Label' as the column name.
"""
categories = labels_df['Label'].unique() # Get unique categories
fig = plt.figure(figsize=(15, 10)) # Define figure size
for i, category in enumerate(categories):
# Find the first image index corresponding to this category
category_indices = labels_df[labels_df['Label'] == category].index
representative_index = category_indices[0]
# Add subplot for this category
ax = fig.add_subplot(3, 4, i + 1)
ax.imshow(images[representative_index]) # Plot the representative image
ax.set_title(category, fontsize=10) # Set the title as the category name
ax.axis('off') # Hide the axis for cleaner visualization
plt.tight_layout()
plt.show()
# Example usage
plot_category_representatives(images, Labels_df)
Observations on Category Differences in Plant Images:¶
Visual Diversity:
Each plant category shows clear visual differences in shape, size, and orientation of leaves. For example:
"Fat Hen": Features small, upright leaves.
"Loose Silky-bent": Thin and grass-like structure.
"Shepherds Purse": Rounded, broader leaves.
Color Differences:
Most plants have green leaves, but the intensity of green varies:
Categories like "Common Chickweed" and "Cleavers" show a darker green.
"Common wheat" and "Loose Silky-bent" have slightly paler green tones.
Leaf Shape and Texture:
Categories differ in the leaf texture:
"Charlock": Broad leaves with noticeable structure.
"Black-grass": Very thin, elongated leaves resembling blades of grass.
"Sugar beet": Large leaves with prominent veins.
Surrounding Features:
- The background (soil, pebbles, and scale markers) is consistent across all images, which helps focus on the plants.
Category Representation:
- The plot is well-structured to showcase one representative image per category, making it easy to identify distinct visual characteristics of each plant.
Practical Insights:
Leaf size, orientation, and texture are the key differentiators between plant types.
Categories like "Loose Silky-bent" and "Black-grass" may be harder to differentiate due to their similar thin leaf shapes.
Checking for data imbalance¶
sns.countplot(x=Labels_df['Label']) # Checking for data imbalance
plt.xticks(rotation='vertical') # Rotating the x-axis labels for better visibility
plt.show()
sns.countplot(Labels_df['Label'])
plt.xticks(rotation='vertical')
Observations on Data Imbalance¶
Based on the countplot of the Label column:
Class Imbalance Exists:
Some classes, such as "Loose Silky-bent", have significantly higher counts (over 600) compared to others.
Classes like "Shepherds Purse", "Cleavers", and "Black-grass" have much fewer samples, indicating underrepresentation.
Major Class:
- The "Loose Silky-bent" class dominates the dataset with the highest count, which might bias the model toward predicting this class more frequently.
Minor Classes:
- Classes such as "Shepherds Purse", "Cleavers", and "Maize" have low sample counts (around 200-300). These classes may struggle to generalize during training.
Balanced Classes:
- Some classes, like "Scentless Mayweed", "Fat Hen", and "Small-flowered Cranesbill", are moderately represented with counts around 400-500.
Risk of Model Bias:
- Due to the imbalance, the model might favor predicting the overrepresented classes (e.g., "Loose Silky-bent"), while it may underperform on the underrepresented ones.
2. Is the dataset provided an imbalance? (Check with using bar plots)?¶
# Count plot to check for data imbalance
sns.countplot(x=Labels_df['Label'], palette='viridis') # Use 'Label' column
plt.title('Class Distribution')
plt.xlabel('Plant Species')
plt.ylabel('Count')
plt.xticks(rotation='vertical') # Rotate labels for better visibility
plt.show()
Observations:¶
Imbalance Exists:
- The dataset has an imbalance in class distribution.
Majority Class:
- 'Loose Silky-bent' has the highest count.
Minority Classes:
- Classes like 'Maize' and 'Shepherd's Purse' have significantly lower counts compared to others.
Distribution Varies:
- Counts are unevenly spread across different plant species.
Data Pre-Processing¶
Resize the images¶
As the size of the images is large, it may be computationally expensive to train on these larger images; therefore, it is preferable to reduce the image size from 128 to 64.
images_decreased = []
height = 64 # Define the height as 64
width = 64 # Define the width as 64
dimensions = (width, height)
for i in range(len(images)):
images_decreased.append(cv2.resize(images[i], dimensions, interpolation=cv2.INTER_LINEAR)) # Resize each image
Image before resizing¶
plt.imshow(images[3])
Image after resizing¶
plt.imshow(images_decreased[3])
Observations:¶
Before Resizing:
- The original image has a higher resolution and retains more detail.
After Resizing:
The image dimensions are reduced to 64x64 pixels.
The resized image appears blurred and pixelated due to loss of resolution.
Visualizing images using Gaussian Blur¶
# Applying Gaussian Blur to denoise the images
images_gb = [] # List to store images after Gaussian blur
for i in range(len(images)):
# Apply Gaussian blur with kernel size (3,3) and sigmaX = 0 (auto-calculated)
images_gb.append(cv2.GaussianBlur(images[i], ksize=(3, 3), sigmaX=0))
Data Preparation for Modeling¶
- Before you proceed to build a model, you need to split the data into train, test, and validation to be able to evaluate the model that you build on the train data
- You'll have to encode categorical features and scale the pixel values.
- You will build a model using the train data and then check its performance
Split the dataset
Labels = Labels_df['Label'].values
X_temp, X_test, y_temp, y_test = train_test_split(np.array(images_decreased), Labels, test_size=0.1, random_state=42, stratify=Labels)
X_train, X_val, y_train, y_val = train_test_split(X_temp, y_temp, test_size=0.1, random_state=42, stratify=y_temp)
print(X_train.shape,y_train.shape)
print(X_val.shape,y_val.shape)
print(X_test.shape,y_test.shape)
Observations on Dataset Splitting:¶
Dataset Size and Splits:
The data has been split into training (80.1%), validation (9%), and test (10%) subsets. This ratio is appropriate for image classification tasks, where:
The training set is large enough for model learning.
- The validation set is sufficient for hyperparameter tuning.
- The test set is representative for final evaluation.
Stratified Sampling:
- Stratification ensures that the class distribution in the train, validation, and test sets is preserved. This is important for balanced learning across all categories and prevents any bias toward specific classes.
Data Shapes:
Each image has dimensions (64, 64, 3), indicating they are resized RGB images.
Training set: 3847 images.
Validation set: 428 images.
Test set: 475 images.
Validation Set Proportion:
- The validation set size of 428 images (~10% of training data) is appropriate for tuning hyperparameters and monitoring overfitting.
Test Set Size:
- A test set of 475 images (~10% of total data) ensures reliable performance evaluation on unseen data.
Encode the target labels¶
# Initialize LabelBinarizer
enc = LabelBinarizer()
# Fit and transform y_train, transform y_val and y_test
y_train_encoded = enc.fit_transform(y_train)
y_val_encoded = enc.transform(y_val)
y_test_encoded = enc.transform(y_test)
# Check the shape of the encoded labels
print(y_train_encoded.shape, y_val_encoded.shape, y_test_encoded.shape)
Observations on Encoding Labels:¶
Label Encoding:
- The LabelBinarizer transforms the target labels into one-hot encoded format, suitable for multi-class classification tasks.
Consistency:
- fit_transform on y_train ensures that the label encoding is learned from the training set.
- transform on y_val and y_test ensures consistency in label encoding across all datasets.
Shapes:
- The output shapes (e.g., (3847, num_classes), (428, num_classes), (475, num_classes)) confirm that labels have been encoded into a 2D one-hot format.
Readiness:
- The data is now ready for use with classification models like neural networks or other algorithms requiring categorical outputs.
Data Normalization¶
X_train_normalized = X_train.astype('float32') / 255.0
X_val_normalized = X_val.astype('float32') / 255.0
X_test_normalized = X_test.astype('float32') / 255.0
Data Normalization Observations:¶
Normalization Process:
Pixel values are divided by 255.0 to scale them from the range [0, 255] to [0, 1].
This ensures the data is in a range suitable for neural networks, improving convergence.
Data Type:
- Conversion to float32 avoids type mismatch issues (originally uint8).
Consistency:
- All datasets (X_train, X_val, X_test) are normalized similarly, ensuring consistent input scales.
Readiness:
- The normalized data is now ready for training models, especially deep learning algorithms.
Model Building¶
# Clearing backend
backend.clear_session()
# Fixing the seed for random number generators
np.random.seed(42)
random.seed(42)
tf.random.set_seed(42)
# Initializing a sequential model
model_1 = Sequential() # Initialize the model
# Adding the first conv layer with 64 filters and kernel size 3x3, padding 'same'
model_1.add(Conv2D(64, (3, 3), activation='relu', padding="same", input_shape=(64, 64, 3)))
# Adding MaxPooling to reduce the size of the output of the first conv layer
model_1.add(MaxPooling2D((2, 2), padding='same'))
# Creating two similar convolution and max-pooling layers with activation = relu
#model1.add(Conv2D(64, (3, 3), activation='relu', padding="same"))
#model1.add(MaxPooling2D((2, 2), padding='same'))
model_1.add(Conv2D(32, (3, 3), activation='relu', padding="same"))
model_1.add(MaxPooling2D((2, 2), padding='same'))
# Flattening the output of the conv layer after max pooling to make it ready for creating dense connections
model_1.add(Flatten()) # Flatten the 2D matrix into 1D
# Adding a fully connected dense layer with 16 neurons
model_1.add(Dense(16, activation='relu'))
model_1.add(Dropout(0.3)) # Dropout layer to avoid overfitting
# Adding the output layer with 12 neurons and softmax activation function for multi-class classification
model_1.add(Dense(12, activation='softmax'))
# Using the Adam optimizer
opt = Adam()
# Compiling the model with categorical cross-entropy loss and accuracy as the metric
model_1.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])
# Generating the summary of the model to see the details
model_1.summary()
Fitting the model_1 on the train data¶
history = model_1.fit(
X_train_normalized, y_train_encoded, # Training data
epochs=30, # Number of epochs (iterations over the dataset)
validation_data=(X_val_normalized, y_val_encoded), # Validation data for monitoring
batch_size=32, # Number of samples per gradient update
verbose=2 # Verbosity level, 2 for a progress bar per epoch
)
Model Evaluation¶
Plotting the training and validation Accuracy¶
plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
Plotting the training and validation loss¶
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.title('Model Loss')
plt.ylabel('Loss')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
Evaluating the model on test data¶
accuracy = model_1.evaluate(X_test_normalized, y_test_encoded, verbose=2)
Validation Accuracy¶
val_loss, val_accuracy = model_1.evaluate(X_val_normalized, y_val_encoded, verbose=2)
# Print Validation Accuracy
print(f"Validation Accuracy: {val_accuracy:.4f}")
Training Accuracy¶
train_loss, train_accuracy = model_1.evaluate(X_train_normalized, y_train_encoded, verbose=2)
print(f"Training Accuracy: {train_accuracy:.4f}")
Generating the predictions using test data¶
# Here we would get the output as probablities for each category
y_pred= model_1.predict(X_test_normalized)
y_pred
model_name = "Base CNN Model"
Plotting the Confusion Matrix¶
# Predicting the output probabilities for each category
y_pred = model_1.predict(X_test) # Predict probabilities on test data
# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg = np.argmax(y_pred, axis=1) # Convert probabilities to class labels
y_test_arg = np.argmax(y_test_encoded, axis=1)
# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg=np.argmax(y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)
# Plotting the Confusion Matrix using confusion matrix() function which is also predefined tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg)
f, ax = plt.subplots(figsize=(10, 8))
sns.heatmap(
confusion_matrix,
annot=True,
linewidths=.4,
fmt="d",
square=True,
ax=ax
)
plt.show()
Observations:¶
Diagonal Values:
- Higher diagonal values (e.g., Class 6, 3, and 10) indicate that these classes are well-predicted.
Misclassifications:
- Off-diagonal values (e.g., Class 0 predicted as Class 6 or Class 9 predicted as others) highlight frequent misclassifications.
Class Imbalance:
- Certain rows have consistently lower counts (e.g., Class 4), indicating either fewer samples or poor model predictions for these classes.
Model Strength:
- Class 6 has the highest correct predictions (strong performance).
# Plotting the Confusion Matrix
confusion_matrix = tf.math.confusion_matrix(y_test_arg, y_pred_arg) # Compute the confusion matrix
# Plot the confusion matrix using Seaborn heatmap
f, ax = plt.subplots(figsize=(12, 12))
sns.heatmap(
confusion_matrix,
annot=True,
linewidths=.4,
fmt="d",
square=True,
ax=ax
)
# Setting the labels to both the axes
ax.set_xlabel('Predicted labels')
ax.set_ylabel('True labels')
ax.set_title('Confusion Matrix')
ax.xaxis.set_ticklabels(list(enc.classes_), rotation=40) # Replace 'enc.classes_' with the actual label encoder classes if needed
ax.yaxis.set_ticklabels(list(enc.classes_), rotation=20) # Replace 'enc.classes_' with the actual label encoder classes if needed
plt.show()
Classification Report¶
# Assuming y_test_arg and y_pred_arg contain the true and predicted class indices, respectively
cr = metrics.classification_report(y_test_arg, y_pred_arg)
# Print the classification report
print(cr)
Observations:¶
- The accuracy is very low at 23%, and both the macro average and weighted average F1-scores are poor (0.13 and 0.16, respectively).
Class Imbalance Issues:
- Several classes (e.g., 0, 2, 5, 7, 9, 10) have precision, recall, and F1-scores of 0, indicating the model fails to predict these classes entirely.
Best Performance on Class 3:
- Class 3 has a recall of 0.98, suggesting that most instances of this class are correctly identified, though precision (0.25) is low, resulting in many false positives.
Moderate Performance on Few Classes:
- Class 1, Class 4, and Class 8 show modest performance but still lack strong precision and recall.
Model Performance Improvement¶
Reducing the Learning Rate:
Hint: Use ReduceLRonPlateau() function that will be used to decrease the learning rate by some factor, if the loss is not decreasing for some time. This may start decreasing the loss at a smaller learning rate. There is a possibility that the loss may still not decrease. This may lead to executing the learning rate reduction again in an attempt to achieve a lower loss.
# Code to monitor val_accuracy
learning_rate_reduction = ReduceLROnPlateau(monitor='val_accuracy',
patience=3,
verbose=1,
factor=0.5,
min_lr=0.00001)
Data Augmentation¶
Remember, data augmentation should not be used in the validation/test data set.
# Clearing backend
backend.clear_session()
# Fixing the seed for random number generators
np.random.seed(42)
random.seed(42)
tf.random.set_seed(42)
# Data Augmentation
train_datagen = ImageDataGenerator(
rotation_range=20, # Rotation angle range to randomly rotate images during training
fill_mode='nearest' # Fill the pixels in the new area created by rotation
)
# Initializing a sequential model
model_2 = Sequential()
# Adding the first convolution layer with 64 filters, kernel size 3x3, and padding 'same'
model_2.add(Conv2D(64, (3, 3), activation='relu', padding="same", input_shape=(64, 64, 3)))
# Adding max pooling to reduce the size of the output of the first convolution layer
model_2.add(MaxPooling2D((2, 2), padding='same'))
# Adding second convolution and max pooling layers
model_2.add(Conv2D(32, (3, 3), activation='relu', padding="same"))
model_2.add(MaxPooling2D((2, 2), padding='same'))
# Adding BatchNormalization for faster convergence
model_2.add(BatchNormalization())
# Flattening the output of the convolution layers after max pooling to make it ready for dense layers
model_2.add(Flatten())
# Adding a fully connected dense layer with 16 neurons
model_2.add(Dense(16, activation='relu'))
# Adding dropout with dropout_rate=0.3 to reduce overfitting
model_2.add(Dropout(0.3))
# Adding the output layer with 12 neurons and softmax activation function for multi-class classification
model_2.add(Dense(12, activation='softmax'))
# Initializing the Adam optimizer
opt = Adam()
# Compiling the model with categorical crossentropy loss function and accuracy as the metric
model_2.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])
# Generating the summary of the model
model_2.summary()
# Defining epochs and batch_size for training
epochs = 30
batch_size = 64
# Training the model on the augmented data
history_0 = model_2.fit(
train_datagen.flow(X_train_normalized, y_train_encoded, batch_size=batch_size, shuffle=False),
epochs=epochs,
steps_per_epoch=X_train_normalized.shape[0] // batch_size,
validation_data=(X_val_normalized, y_val_encoded),
verbose=1,
callbacks=[learning_rate_reduction]
)
Plotting the training and validation Accuracy¶
plt.plot(history_0 .history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
Plotting the training and validation loss¶
# Plotting the training and validation loss
plt.plot(history_0 .history['loss'])
plt.plot(history.history['val_loss'])
plt.title('Model Loss')
plt.ylabel('Loss')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
Evaluate the model on test data¶
accuracy = model_2.evaluate(X_test_normalized, y_test_encoded, verbose=2)
Validation Accuracy¶
val_loss, val_accuracy = model_2.evaluate(X_val_normalized, y_val_encoded, verbose=2)
# Print Validation Accuracy
print(f"Validation Accuracy: {val_accuracy:.4f}")
Training Accuracy:¶
train_loss, train_accuracy = model_2.evaluate(X_train_normalized, y_train_encoded, verbose=2)
print(f"Training Accuracy: {train_accuracy:.4f}")
model_name = "CNN Model with Data Augmentation"
Generating the predictions using test data¶
# Here we would get the output as probablities for each category
y_pred=model_2.predict(X_test_normalized)
y_pred
Plotting the Confusion Matrix¶
# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg=np.argmax(y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)
# Plotting the Confusion Matrix using confusion matrix() function which is also predefined tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg)
f, ax = plt.subplots(figsize=(10, 8))
sns.heatmap(
confusion_matrix,
annot=True,
linewidths=.4,
fmt="d",
square=True,
ax=ax
)
plt.show()
Observations:¶
Dominant Predictions for Certain Classes:
The model performs relatively well for Class 6 (57 correct predictions) and Class 3 (54 correct predictions).
These classes seem to have better recall compared to others.
Misclassifications:
Class 0 is heavily misclassified, with many instances incorrectly predicted as Class 6 (22 instances).
Similar issues exist for other classes, such as Class 7 and Class 5, with predictions scattered across other classes.
Class Imbalance Impact:
- Certain classes (e.g., 0, 2, 7) have relatively fewer correct predictions, indicating potential class imbalance or insufficient representation in the model.
Diagonal Dominance:
- While the diagonal has some high values for specific classes, many off-diagonal elements indicate frequent misclassifications.
Plotting the Confusion Matrix with Labels¶
# Complete the code to obtain the output probabilities
y_pred = model_2.predict(X_test_normalized) # Predicting the output probabilities for test data
# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg = np.argmax(y_pred, axis=1) # Converting probabilities to categorical predictions
y_test_arg = np.argmax(y_test_encoded, axis=1) # Converting true labels to categorical values
# Plotting the Confusion Matrix using confusion_matrix() function which is predefined in tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg, y_pred_arg) # Generating the confusion matrix
f, ax = plt.subplots(figsize=(12, 12))
sns.heatmap(
confusion_matrix,
annot=True, # Annotating the confusion matrix with the numbers
linewidths=.4,
fmt="d", # Displaying the values as integers
square=True,
ax=ax
)
# Setting the labels to both the axes
ax.set_xlabel('Predicted labels')
ax.set_ylabel('True labels')
ax.set_title('Confusion Matrix')
ax.xaxis.set_ticklabels(list(enc.classes_), rotation=40) # Assuming `enc.classes_` has the class names
ax.yaxis.set_ticklabels(list(enc.classes_), rotation=20)
plt.show()
Plotting Classification Report¶
# Complete the code to generate the classification report
cr = metrics.classification_report(y_test_arg, y_pred_arg) # Generate the classification report
# Print the classification report
print(cr)
Observations:¶
Overall Accuracy:
- The model achieves an overall accuracy of 74%, which indicates reasonably good performance but with room for improvement.
Class Performance:
Strong Performance:
- Classes 1, 3, 5, 10, and 11 have precision, recall, and F1-scores close to or above 0.80, indicating strong model performance for these classes.
Weak Performance:
Class 0 has the lowest performance, with a precision of 0.33 and a recall of 0.12, suggesting that it is difficult for the model to identify and correctly predict this class.
Class 4 also struggles, with very low recall (0.05) and an F1-score of 0.08, indicating poor detection of this class.
Recall vs. Precision:
High Recall: Class 6 has a high recall of 0.88 but a lower precision of 0.55, suggesting that while the model is good at detecting this class, it also produces many false positives.
Balanced Metrics: Classes 3, 1, and 11 show a good balance between precision and recall, leading to high F1-scores.
Macro vs. Weighted Average:
- The macro average scores are slightly lower than the weighted average scores, which indicates that the model’s performance is uneven across different classes, with some classes dragging down the average.
Transfer Learning using VGG16¶
# Load the pre-trained VGG16 model without the top layer
vgg_model = VGG16(weights='imagenet', include_top=False, input_shape=(64, 64, 3))
# Freeze all the layers in the VGG16 model
for layer in vgg_model.layers:
layer.trainable = False
# Build your custom model
new_model = Sequential()
# Add the VGG16 convolutional base
new_model.add(vgg_model)
# Add custom layers on top of the VGG16 base
new_model.add(Flatten())
new_model.add(Dense(32, activation='relu'))
new_model.add(Dropout(0.2))
new_model.add(Dense(16, activation='relu'))
# Calculate num_classes
num_classes = y_train_encoded.shape[1] # Ensure y_train_encoded is properly encoded
# Add the output layer
#new_model.add(Dense(10, activation='softmax'))
new_model.add(Dense(num_classes, activation='softmax'))
# Compile the model
opt = Adam()
new_model.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])
# Verify the model structure
new_model.summary()
# Train the model
epochs = 30
batch_size = 64
history_vgg16 = new_model.fit(
train_datagen.flow(X_train_normalized, y_train_encoded, batch_size=batch_size, seed=42, shuffle=False),
epochs=epochs,
steps_per_epoch=X_train_normalized.shape[0] // batch_size,
validation_data=(X_val_normalized, y_val_encoded),
verbose=1
)
Plotting the training and validation Accuracy¶
# Plot accuracy
plt.plot(history_vgg16.history['accuracy'], label='Train Accuracy')
plt.plot(history_vgg16.history['val_accuracy'], label='Validation Accuracy')
plt.legend()
plt.show()
Plotting the training and validation loss¶
# Plot loss
plt.plot(history_vgg16.history['loss'], label='Train Loss')
plt.plot(history_vgg16.history['val_loss'], label='Validation Loss')
plt.title('Training and Validation Loss')
plt.xlabel('Epochs')
plt.ylabel('Loss')
plt.legend()
plt.show()
Evaluate the model on test data¶
accuracy = new_model.evaluate(X_test_normalized, y_test_encoded, verbose=2)
loss, accuracy = new_model.evaluate(X_test_normalized, y_test_encoded, verbose=2)
print(f"Test Loss: {loss}")
print(f"Test Accuracy: {accuracy}")
Validation Accuracy¶
val_loss, val_accuracy = new_model.evaluate(X_val_normalized, y_val_encoded, verbose=2)
# Print Validation Accuracy
print(f"Validation Accuracy: {val_accuracy:.4f}")
Training Accuracy¶
train_loss, train_accuracy = new_model.evaluate(X_train_normalized, y_train_encoded, verbose=2)
print(f"Training Accuracy: {train_accuracy:.4f}")
# Model name
model_name = "Transfer Learning Model"
Generating the predictions using test data¶
y_pred = new_model.predict(X_test_normalized)
y_pred
Confusion Matrix plot¶
# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg=np.argmax(y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)
# Plotting the Confusion Matrix using confusion matrix() function which is also predefined tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg)
f, ax = plt.subplots(figsize=(10, 8))
sns.heatmap(
confusion_matrix,
annot=True,
linewidths=.4,
fmt="d",
square=True,
ax=ax
)
plt.show()
Confusion Matrix with labels plots¶
Observations:¶
Improvement in Class 6:
- Class 6 continues to perform the best, with 60 correct predictions, indicating strong model performance for this class.
Misclassification Patterns:
Class 0 is misclassified often as Class 3 (16 times), highlighting a need to distinguish these two classes better.
Class 5 has significant misclassifications across multiple classes, with only 16 correctly predicted.
Class 3 Performance:
- Class 3 shows some improvement with 36 correct predictions but still has misclassifications distributed across other classes.
Diagonal Dominance:
- The diagonal values show improvement in certain classes but remain low for others, such as Classes 0, 2, and 7.
# Complete the code to obtain the output probabilities
y_pred = new_model.predict(X_test_normalized) # Predicting the output probabilities for test data
# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg = np.argmax(y_pred, axis=1) # Converting probabilities to categorical predictions
y_test_arg = np.argmax(y_test_encoded, axis=1) # Converting true labels to categorical values
confusion_matrix = tf.math.confusion_matrix(y_test_arg, y_pred_arg) # Generating the confusion matrix
f, ax = plt.subplots(figsize=(12, 12))
sns.heatmap(
confusion_matrix,
annot=True, # Annotating the confusion matrix with the numbers
linewidths=.4,
fmt="d", # Displaying the values as integers
square=True,
ax=ax
)
# Setting the labels to both the axes
ax.set_xlabel('Predicted labels')
ax.set_ylabel('True labels')
ax.set_title('Confusion Matrix')
ax.xaxis.set_ticklabels(list(enc.classes_), rotation=40)
ax.yaxis.set_ticklabels(list(enc.classes_), rotation=20)
plt.show()
Classification report¶
# Assuming y_test_arg and y_pred_arg contain the true and predicted class indices, respectively
cr = metrics.classification_report(y_test_arg, y_pred_arg)
# Print the classification report
print(cr)
Observations:¶
Overall Accuracy:
- The model achieves an accuracy of 53%, indicating moderate performance but with significant room for improvement.
Class-Level Observations:
Strong Performance:
Class 6 has the highest recall (0.92) and a good F1-score (0.71), showing the model's strong ability to detect instances of this class.
Class 7 has a high precision (0.85), but its recall is only 0.50, indicating good specificity but struggles in identifying all true positives.
Weak Performance:
Class 4 has zero scores for precision, recall, and F1, indicating that the model completely fails to predict this class.
Classes 9 and 0 also perform poorly, with low recall and F1-scores, highlighting challenges in identifying these classes.
Imbalance Between Precision and Recall:
Some classes, such as 10, have significantly better recall (0.74) than precision (0.40), suggesting many false positives.
Conversely, classes like 7 have higher precision but lower recall, indicating the opposite issue.
Macro vs. Weighted Averages:
- The macro average F1-score (0.46) is lower than the weighted average (0.50), reflecting uneven model performance across classes, where poorly performing classes impact the macro score more.
Final Model¶
Comment on the final model you have selected and use the same in the below code to visualize the image.¶
Final model_name "CNN Model with Data Augmentation"¶
accuracy = model_2.evaluate(X_test_normalized, y_test_encoded, verbose=2)
val_loss, val_accuracy = model_2.evaluate(X_val_normalized, y_val_encoded, verbose=2)
# Print Validation Accuracy
print(f"Validation Accuracy: {val_accuracy:.4f}")
train_loss, train_accuracy = model_2.evaluate(X_train_normalized, y_train_encoded, verbose=2)
print(f"Training Accuracy: {train_accuracy:.4f}")
# Here we would get the output as probablities for each category
y_pred=model_2.predict(X_test_normalized)
y_pred
# Plotting the Confusion Matrix using confusion matrix() function which is also predefined tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg)
f, ax = plt.subplots(figsize=(10, 8))
sns.heatmap(
confusion_matrix,
annot=True,
linewidths=.4,
fmt="d",
square=True,
ax=ax
)
plt.show()
Visualizing the prediction¶
# Visualizing the predicted and correct label of images from test data
plt.figure(figsize=(2,2))
plt.imshow(X_test[2])
plt.show()
# Complete the code to predict the test data using the final model selected
print('Predicted Label', enc.inverse_transform(model_2.predict((X_test_normalized[2].reshape(1,64,64,3))))) # reshaping the input image as we are only trying to predict using a single image
print('True Label', enc.inverse_transform(y_test_encoded)[2]) # using inverse_transform() to get the output label from the output vector
plt.figure(figsize=(2,2))
plt.imshow(X_test[33])
plt.show()
# Complete the code to predict the test data using the final model selected
print('Predicted Label', enc.inverse_transform(model_2.predict((X_test_normalized[2].reshape(1,64,64,3))))) # reshaping the input image as we are only trying to predict using a single image
print('True Label', enc.inverse_transform(y_test_encoded)[33]) # using inverse_transform() to get the output label from the output vector
plt.figure(figsize=(2,2))
plt.imshow(X_test[59])
plt.show()
# Complete the code to predict the test data using the final model selected
print('Predicted Label', enc.inverse_transform(model_2.predict((X_test_normalized[2].reshape(1,64,64,3))))) # reshaping the input image as we are only trying to predict using a single image
print('True Label', enc.inverse_transform(y_test_encoded)[59]) # using inverse_transform() to get the output label from the output vector
plt.figure(figsize=(2,2))
plt.imshow(X_test[36])
plt.show()
# Complete the code to predict the test data using the final model selected
print('Predicted Label', enc.inverse_transform(model_2.predict((X_test_normalized[2].reshape(1,64,64,3))))) # reshaping the input image as we are only trying to predict using a single image
print('True Label', enc.inverse_transform(y_test_encoded)[36]) # using inverse_transform() to get the output label from the output vector
Observations from the Final Model Evaluation¶
Performance Metrics:
Training Accuracy: 78.42%
Validation Accuracy: 73.13%
Test Accuracy: 73.68%
The model generalizes well, as the gap between training, validation, and test accuracies is minimal, indicating no overfitting.
Prediction Probabilities:
The model outputs probabilities for each category, suggesting it is suitable for multi-class classification.
Higher probabilities for correct classes demonstrate the model's confidence.
Confusion Matrix:
Diagonal Dominance: The matrix's diagonal contains the highest values, indicating accurate predictions for most classes.
Misclassifications: Off-diagonal entries represent misclassified samples.
Certain categories may need further augmentation or fine-tuning.
Balanced Performance: No significant bias toward any specific class, suggesting improved handling of class imbalance.
Improvements Over Other Models:
- The "CNN Model with Data Augmentation" outperforms the base and transfer learning models in all metrics, including accuracy and balanced class predictions.
Prediction Consistency Issue:
All the images are predicted as 'Small-flowered Cranesbill' irrespective of their true labels:
True Label: Small-flowered Cranesbill → Correct Prediction
True Label: Cleavers, Common Chickweed, Shepherds Purse → Incorrect Prediction.
Model Bias:
- The model seems biased toward predicting 'Small-flowered Cranesbill', possibly due to imbalanced data or poor generalization.
Evaluation of Predictions:
- The test images include clearly distinct leaf structures, yet the model fails to differentiate between them. This indicates issues with feature extraction or insufficient training.
pd.DataFrame({'Models':['Base CNN Model','CNN Model with Data Augmentation','Transfer Learning Model'],'Train Accuracy':['61%','78%','61%'],'Validation Accuracy':['53%','73%','50%'],'Test Accuracy':['51%','74%','53%']})
Observations¶
We can observe from the confusion matrix of all the models that our CNN Model with Data Augmentation model was the best model because it predicted the majority of the classes better than the other models.
The test accuracy of the CNN Model with Data Augmentation model is 73%. Data Augmentation has also helped in improving the model.
Conclusion¶
Conclusion¶
- The CNN model with data augmentation is the most viable option for deployment due to its superior accuracy and generalization. While the base model and transfer learning approach fail to deliver acceptable performance, further experimentation with data augmentation and domain-specific pre-trained models may enhance results.
Actionable Insights and Business Recommendations¶
1. Base CNN Model (Train Accuracy: 61%, Validation Accuracy: 53%, Test Accuracy: 51%)¶
Performance:
- The model underperforms with a training accuracy of 61%, validation accuracy of 53%, and test accuracy of 51%.
Insight:
- The model struggles with both overfitting and underfitting, failing to generalize beyond the training set. Its low accuracy on all datasets indicates it might lack sufficient complexity or proper tuning.
2. CNN Model with Data Augmentation (Train Accuracy: 78%, Validation Accuracy: 73%, Test Accuracy: 74%)¶
Performance:
- The model achieves the best performance, with a training accuracy of 78%, validation accuracy of 73%, and test accuracy of 74%.
Insight:
- Data augmentation effectively improves generalization, allowing the model to learn better representations and perform well on unseen data. This indicates that the dataset benefits significantly from augmentation.
3. Transfer Learning Model (Train Accuracy: 61%, Validation Accuracy: 50%, Test Accuracy: 53%)¶
Performance:
- Despite leveraging pre-trained features, this model exhibits poor performance, with training accuracy at 61%, validation at 50%, and test at 53%.
Insight:
- The transfer learning approach is not effective for this dataset, potentially due to a mismatch between the pre-trained model's domain and the dataset's characteristics.
Business Recommendations¶
1.Adopt the CNN Model with Data Augmentation¶
Reason:
- It delivers the highest accuracy across all datasets, indicating robust performance and reliability for deployment.
2. Enhance Data Augmentation¶
- Experiment with different augmentation techniques (e.g., rotations, zooms, flips, color shifts) to explore further performance gains.
3. Review the Transfer Learning Strategy¶
- Evaluate the choice of pre-trained models and consider alternatives better suited to the dataset's domain. For instance, selecting models pre-trained on similar data types could improve performance.
4. Hyperparameter Tuning¶
- Conduct hyperparameter optimization on the CNN with Data Augmentation model to explore potential for further accuracy improvements.
5. Monitor Real-World Performance¶
- Post-deployment, track key performance indicators (KPIs) to ensure the model meets business expectations and adjust training as needed.